Dykstra's Algorithm, ADMM, and Coordinate Descent: Connections, Insights, and Extensions

Supplement to: “Dykstra’s Algorithm, ADMM, and Coordinate Descent: Connections, Insights, and Extensions”

Penalized Bregman Divergence Estimation via Coordinate Descent

Variable selection via penalized estimation is appealing for dimension reduction. For penalized linear regression, Efron, et al. (2004) introduced the LARS algorithm. Recently, the coordinate descent (CD) algorithm was developed by Friedman, et al. (2007) for penalized linear regression and penalized logistic regression and was shown to gain computational superiority. This paper explores...

Cis400/401 Final Project Report

The rise of ‘big data’ and large-scale machine learning has created an increasing need for distributed optimization.dual descent and the alternating Most of the current literature has focused on coordinate descent, a prominent distributed optimization technique, due to its simplicity and effectiveness. We focus on implementing two other optimization techniques distributed dual descent and the a...

Continuous Relaxation of MAP Inference: A Nonconvex Perspective

In this paper, we study a nonconvex continuous relaxation of MAP inference in discrete Markov random fields (MRFs). We show that for arbitrary MRFs, this relaxation is tight, and a discrete stationary point of it can be easily reached by a simple block coordinate descent algorithm. In addition, we study the resolution of this relaxation using popular gradient methods, and further propose a more...

A Local Analysis of Block Coordinate Descent for Gaussian Phase Retrieval

While convergence of the Alternating DirectionMethod of Multipliers (ADMM) on convex problems is well studied, convergence on nonconvex problems is only partially understood. In this paper, we consider the Gaussian phase retrieval problem, formulated as a linear constrained optimization problem with a biconvex objective. The particular structure allows for a novel application of the ADMM. It ca...